Digital still camera and image correction method

ABSTRACT

To be capable of correcting a captured image to an image desirable to a user only by using a digital still camera. 
     An RAM  15  storing the image data of an image for which tone correction is to be performed in a standard color space format and a tone correction circuit  35  are provided. The image data is read out from the RAM  15 , and the tone correction is performed for the readout image data in the tone correction circuit  35 . In automatic tone correction, luminance signals of the image are statistically analyzed to categorize the image, and an appropriate correction curve is used to correct the image.

TECHNICAL FIELD

The present invention relates to a digital still camera and an imagecorrection method.

BACKGROUND ART

As seen from the fact that the total volume of shipments of digitalstill cameras exceeds that of film cameras, the digital still camerashave increasingly been in widespread use. Along this popularization,various levels of users, from high-end users to low-end users, use thedigital still cameras and, therefore, the application of the digitalstill cameras, the required level of the image quality, the preferencein the image quality, and so on have become diversified.

For example, users who have used film cameras and are familiar with theimage quality of the film cameras tend to prefer “film-camera-like”images when the users compare images captured by digital still cameraswith pictures taken by film cameras. In contrast, users who arealienated from images taken by film cameras or still pictures and arefamiliar to the image quality on TV screens tend to prefer“television-like” image qualities when the users compare images on TVscreens with images captured by digital still cameras.

Hence, the manufacturers of the digital still cameras are devisingimages for every camera type and are developing the image qualities withwhich assumed target users seem to be satisfied.

As described above, as the user group of digital still cameras becomeslarge, the preference in the image qualities is diversified. It isconsiderably difficult in the current state to realize the imagequalities satisfying the preferences of the users in one digital stillcamera. This is because color images captured by digital still camerasare subjected to AWB control (automatic white balance control), tonecorrection, saturation correction, and so on based on the knowhow uniqueto each of the manufacturers of the digital still cameras and, then, arerecorded in flash memories.

A format compliant with DCF is generally used as an image format whenimage data is recorded in a flash memory. An sRGB color space, which isa standard color space, is adopted in such a format. In the sRGB colorspace, tone characteristics or color characteristics including a colorgamut (a color reproduction range) of the CRT monitors of personalcomputers are defined.

In addition, JPEG2000, which is adopted in the DCF and which is anextension of JPEG format, and color spaces that can widely cover thecolor reproduction range perceived by human beings are considered asimage formats. Among these formats or color spaces, an scRGB color space(scene-reference color space) in which the amount of light of a realscene can be stored as linear data, and/or a color space(output-reference color space), for example, an extended sRGB colorspace, in which the color of data is corrected so as to be appropriatefor a destination or a display device, have been internationallystandardized.

However, whichever image format is selected, since captured color imagesare recorded and stored in a flash memory as a simple image in the imageformat compliant with the DCF, the images satisfying all the imagequalities required by various users cannot be stored.

There is another problem in that a captured image becomes a so-calledfailed image due to an error in setting the basic conditions in theimage capture. For example, the color balance of an image can bederogated or the image can be underexposed because of the image capturein a natural light mode below fluorescent light. However, since digitalimage data is captured in digital still cameras, unlike the images takenby film cameras, there is more need for modification after the imagecapture.

Countermeasures against the above problems include a method of utilizingcommercially available “image correction and processing software” in apersonal computer. If a user owns not only a digital still camera butalso a personal computer, images captured by the digital still cameracan be corrected by using such software to produce appropriate images.

However, the image data which a user can use in the correction isactually data subjected to JPEG compression or the like in the digitalstill camera. Since the amount of information in the image data issmaller than that of an actual scene, the image cannot necessarily becorrected to the image quality with which the user, particularly, ahigh-end user, is satisfied.

It is difficult for and troublesome to a low-end user to correct thequality of a captured image to the image quality for which he/she has apreference by the use of the correction and processing software.Accordingly, almost all the commercially available correction andprocessing software has an “image-quality automatic correctionfunction”, and even a user who has no knowledge of the image correctioncan simply correct the image. However, under the current conditions, thecorrection effect can be varied depending on the content of the imageand, therefore, a desired image quality cannot often be achieved.

Furthermore, there is a problem of storage in a current sRGB format or aJPEG-YCC format compliant with the current sRGB format in the correctionafter the image capture. As for the method of storing the image in ascene-reference color space format, such as an scRGB color space or anscYCC color space, a solution is proposed in, for example, JapaneseUnexamined Patent Application Publication No. 2001-343753. However,there is currently no image correction software accommodated to theimage in the scene-reference color space format.

In order to resolve the problems described above, the present inventionprovides an image correction method in which a user can correct acaptured image to an appropriate or desired image and which can beperformed in a digital still camera.

DISCLOSURE OF INVENTION

The present invention provides, for example, a digital still cameraincluding a memory that stores the image data of an image for which tonecorrection is to be performed in a standard color space format and atone correction circuit. The image data is read out from the memory, andthe tone correction is performed for the readout image data in the tonecorrection circuit.

With this structure, the image is corrected to an image desirable to auser in the digital still camera.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing an embodiment of a capturingsystem according to the present invention.

FIG. 2 is a schematic diagram showing an embodiment of a monitoringsystem according to the present invention.

FIG. 3 is a schematic diagram showing an embodiment of a main part inthe monitoring system.

FIG. 4 is a schematic diagram showing an embodiment of a main part inthe monitoring system.

FIG. 5 includes diagrams illustrating examples of GUI operations in themonitoring system.

FIG. 6 includes diagrams illustrating examples of other GUI operationsin the monitoring system.

FIG. 7 is a schematic diagram showing an embodiment of a main part inthe monitoring system.

FIG. 8 is a graph showing a characteristic of a main part in themonitoring system.

FIG. 9 illustrates categories into which captured images arecategorized.

FIG. 10 illustrates an algorithm of a main part in the monitoringsystem.

FIG. 11 illustrates an algorithm of a main part in the monitoringsystem.

FIG. 12 is a graph showing a characteristic of the main part in themonitoring system.

FIG. 13 includes graphs showing characteristics of a main part in themonitoring system.

FIG. 14 is a graph showing characteristics of a main part in themonitoring system.

FIG. 15 is a table showing parameters, provided in the monitoringsystem.

FIG. 16 illustrates an algorithm of main parts in the monitoring system.

FIG. 17 illustrates an algorithm of main parts in the monitoring system.

FIG. 18 is a graph showing characteristics of a main part in themonitoring system.

FIG. 19 illustrates an algorithm of main parts in the monitoring system.

FIG. 20 illustrates an algorithm of main parts in the monitoring system.

FIG. 21 includes mathematical expressions.

FIG. 22 includes mathematical expressions.

FIG. 23 includes mathematical expressions.

FIG. 24 includes mathematical expressions.

FIG. 25 includes mathematical expressions.

FIG. 26 includes mathematical expressions.

BEST MODE FOR CARRYING OUT THE INVENTION 1. Signal Processing in ImageCapture

FIG. 1 shows an example of the image capturing system in a 1CCD digitalstill camera according to the present invention. Specifically, an imageof an object OBJ is projected on a CCD image sensor 11 through animaging lens LNS. Sixteen-bit linear image data R1, G1, and B1corresponding to three primary colors is extracted from the CCD imagesensor 11. The extracted image data R1, G1, and B1 is supplied to ademosaic processing circuit 12 to be demosaiced into image data forevery pixel. The image data is then subjected to AWB processing in anAWB processing circuit 13 to provide image data R′2, G′2, and B′2.

The image data R′2, G′2, and B′2 is supplied to an scRGB data generatingcircuit 14 to provide 16-bit linear image data RscRGB, GscRGB, andBscRGB in an scRGB format, and the image data RscRGB, GscRGB, and BscRGBis temporarily stored in a RAM 15. The stored image data RscRGB, GscRGB,and BscRGB is supplied to a gamma correction circuit 16 to be subjectedto gamma correction and to provide eight-bit image data R, G, and B forthe three colors. The image data R, G, and B is written in nonvolatilememory means, for example, a flash memory 20, through a reader-writer 19and is stored in the flash memory 20. The flash memory 20 is removablefrom the digital still camera.

Alternatively, the image data R, G, and B is supplied from the gammacorrection circuit 16 to a matrix calculating circuit 17 to be convertedinto image data in a YCC format, that is, luminance data Y and colordifference data Cb and Cr of blue and red. The image data Y, Cb, and Cris written in the flash memory 20 through the reader-writer 19 and isstored in the flash memory 20.

The image data R, G, and B or the image data Y, Cb, and Cr is stored inthe flash memory 20 in order to establish the compatibility with a knowndigital still camera or “image correction and processing software”.According to the present invention, the image data is further processedin the following manner.

The image data RscRGB, GscRGB, and BscRGB stored in the RAM 15, that is,the 16-bit linear image data, is written in the flash memory 20 throughthe reader-writer 19 without any modification and is stored in the flashmemory 20.

Alternatively, the image data RscRGB, GscRGB, and BscRGB is supplied toan scYCC/scRGB converting circuit 18 to be converted into 12-bitnonlinear image data YscYCC, CbscYCC, and CrscYCC in an scYCC format.The image data YscYCC, CbscYCC, and CrscYCC is written in the flashmemory 20 through the reader-writer 19 and is stored in the flash memory20.

The signal processing in image capture is performed in the mannerdescribed above.

2. Example of scRGB-Format Image Data Generating Circuit

In the scRGB data generating circuit 14, the image data R′2, G′2, andB′2, which is subjected to the AWB correction, is converted into the16-bit linear image data RscRGB, GscRGB, and BscRGB in the scRGB format.This conversion is realized in, for example, the following process.

(1). First, a white level (100% white level) of a scene is determined.For example, image data Xraw, Yraw, and Xraw is calculated from theimage data R′2, G′2, and B′2, which is subjected to the AWB correction,for every pixel according to Equation 1 in FIG. 21.

Next, the average Yraw_ave of the image data Yraw for all the pixels inthe image or for a pixel appropriately sampled is calculated, and thewhite level is set to a value, for example, five times larger than theaverage Yraw_ave.

(2) The image data Xraw, Yraw, and Xraw for every pixel is normalizedaccording to Equation 2 in FIG. 21 based on the white level set in (1)to calculate normalized image data Xraw_n, Yraw_n, and Xraw_n.(3) As shown by Equation 3 in FIG. 21, the normalized image data Xraw_n,Yraw_n, and Xraw_n calculated in (2) is integrated with the inversematrix of a matrix M1 to calculate image data R′3, G′3, and B′3 forevery pixel.(4) Sixteen-bit linear image data RscRGB, GscRGB, and BscRGB in thescRGB format for every pixel is generated from the image data R′3, G′3,and B′3 calculated in (3), according to Equation 4 in FIG. 21.

Since the image data RscRGB, GscRGB, and BscRGB is desired image data inthe scRGB format, the image data RscRGB, GscRGB, and BscRGB is read outfrom the scRGB data generating circuit 14 and is stored in the RAM 15.

3. Example of scYCC/scRGB Converting Circuit 18

In the scYCC/scRGB converting circuit 18, the image data RscRGB, GscRGB,and BscRGB in the scRGB format is converted into the image data in thescYCC format. This conversion is realized by, for example, the followingprocess.

(1) Nonlinear image data R′scRGB, G′scRGB, and B′scRGB in the scRGBformat is calculated from the 16-bit linear image data RscRGB, GscRGB,and BscRGB in the scRGB format according to Equation 5 in FIG. 21 andEquation 6 in FIG. 22.(2) The nonlinear image data R′scRGB, G′scRGB, and B′scRGB in the scRGBformat is converted into the image data YscYCC, CbscYCC, and CrscYCC inthe scYCC format according to Equations 7 and 8 in FIG. 22.

Since the image data YscYCC, CbscYCC, and CrscYCC is desired image datain the scYCC format, the image data YscYCC, CbscYCC, and CrscYCC isstored in the flash memory 20 through the reader-writer 19.

4. Signal Processing in Monitoring of Captured Image

FIG. 2 shows an example in which the present invention is applied to acircuit for correcting a captured image. Specifically, the image dataRscRGB, GscRGB, and BscRGB in the scRGB format is read out from theflash memory 20 through the reader-writer 19, and the readout image dataRscRGB to BscRGB is written in the RAM 15. Alternatively, the image dataYscYCC, CbscYCC, and CrscYCC in the scYCC format is read out from theflash memory 20 through the reader-writer 19. The image data YscYCC,CbscYCC, and CrscYCC is supplied to the scYCC/scRGB converting circuit18 to be converted into the image data RscRGB, GscRGB, and BscRGB in thescRGB format. The image data RscRGB, GscRGB, and BscRGB is written inthe RAM 15.

The image data RscRGB, GscRGB, and BscRGB written in the RAM 15 issupplied to a display, for example, an LCD 32, through a monitor displayprocessing circuit 31 and is displayed as a color image.

The white balance of the image data stored in the flash memory 20 hasbeen automatically controlled by the AWB processing circuit 13 in theimage capture. In the example in FIG. 2, the image data written in theRAM 15 is processed in an AWB fine tuning circuit 33 to fine-tune thewhite balance of the color image displayed in the LCD 32.

The image data RscRGB, GscRGB, and BscRGB written in the RAM 15 isconverted into the 12-bit nonlinear image data YscYCC, CbscYCC, andCrscYCC in the scYCC format in the scYCC/scRGB converting circuit 18 andis written in a RAM 34. The image data YscYCC, CbscYCC, and CrscYCC inthe RAM 34 is corrected in accordance with a GUI operation by a user ina tone correction circuit 35. The corrected image data is converted intosignals in an RGB format in the monitor display processing circuit 31,and the converted signals are supplied to the LCD 32 and are displayedas a color image.

The image data that is subjected to the tone correction in the tonecorrection circuit 35 is written in the flash memory 20 through thereader-writer 19 and is stored in the flash memory 20.

5. Example of AWB Fine Tuning Circuit 33

The white balance of the image data RscRGB, GscRGB, and BscRGB in thescRGB format, written in the RAM 15, is fine-tuned based on the whiteinformation in the image capture, stored in the digital still camera.This fine tuning is realized by the following process.

5-1 ΔT[K] Correction of Color Temperature from White Point in ImageCapture (Refer to FIG. 3)

5-1-1 Fine Tuning of White Balance on Site After Image Capture

(1) A white-point color-temperature evaluation value Tw[K] of a scene isyielded from the data in the image capture. A white-pointcolor-temperature evaluation value Tw′[K] of the scene after the whitebalance is corrected is calculated from the evaluation value Tw[K]according to Equation 9 in FIG. 22 in a white-point color-temperatureevaluator 331.(2) Xy chromaticities xd′ and yd′ are calculated from the white-pointcolor-temperature information Tw′[K] calculated in (1), according toEquation 10 in FIG. 23. It is known that Equation 10 is approximatelysatisfied at a color temperature of 4,000 K to 7,000 K.(3) XYZ tristimulus values Xw′, Yw′, and Zw′ are calculated from the xychromaticities xd′ and yd′ of the white point, calculated in (2),according to Equation 11 in FIG. 23.(4) Linear RGB values R′w, G′w, and B′w in use for displaying in an sRGBmonitor are calculated from the tristimulus values Xw′, Yw′, and Zw′ ofthe white point after the correction by using the inverse matrix of thematrix M1 used in Equation (1), according to Equation 12 in FIG. 23.(5) With regard to the white point of the Tw[K] before the correction,XYZ tristimulus values Xw, Yw, and Zw can be calculated according toEquations 10 and 11.(6) Linear RGB values Rw, Gw, and Bw when the white point of the scenebefore the correction is displayed in the sRGB monitor are calculatedaccording to Equation 12.(7) White-balance control factors kr, kg, and kb are calculated from thelinear RGB values Rw, Gw, and Bw before the correction and the linearRGB values R′w, G′w, and B′w after the correction according to Equation13 in FIG. 23 in a white-balance control-factor calculator 332.(8) In a white-balance control calculator 333, the white-balance controlfactors kr, kg, and kb calculated in (7) are integrated with the imagedata RscRGB, GscRGB, and BscRGB in the scRGB format, read out from theRAM 15, according to Equation 14 in FIG. 23, to calculate image dataRscRGB_W, GscRGB_W, and BscRGB_W in the scRGB format after thecorrection. The calculated image data RscRGB_W, GscRGB_W, and BscRGB_Wis written back in the RAM 15 as fine-tuned results.

5-1-2 Fine Tuning of White Balance of Image Recorded in Flash Memory 20

(1) White-point information in the image capture, recorded in advance ina header of the captured data (data scRGB or scYCC), is read out toyield the white-point color temperature Tw[K] in the image capture. Thewhite-balance control factors kr, kg, and kb are calculated based onthis information. The white-balance control factors kr, kg, and kb arecalculated in the same manner as in 5-1-1.(2) When the image data stored in the flash memory 20 is the 16-bitlinear image data RscRGB, GscRGB, and BscRGB in the scRGB format, thecorrection is performed in the same manner as in 5-1-1 (6).(3) When image data stored in the flash memory 20 is the 12-bitnonlinear image data YscYCC, CbscYCC, and CrscYCC in the scYCC format,the image data YscYCC, CbscYCC, and CrscYCC is converted into the 16-bitlinear image data RscRGB, GscRGB, and BscRGB in the scRGB format in thescYCC/scRGB converting circuit 18, and the correction is performed inthe same manner as in 5-1-1 (6).

The conversion method is described in detail next.

(3)-1 The 12-bit nonlinear image data YscYCC, CbscYCC, and CrscYCC inthe scYCC format, read out from the flash memory 20, is converted intothe nonlinear image data R′scRGB, G′scRGB, and B′scRGB in the scRGBformat according to Equations 15 and 16 in FIG. 23.(3)-2 The image data R′scRGB, G′scRGB, and B′scRGB converted in (3)-1 isconverted into the linear image data R′3, G′3, and B′3 in the scRGBformat according to Equation 17 in FIG. 24.(3)-3 The image data R′3, G′3, and B′3 converted in (3)-2 is convertedinto 16-bit linear image data RscRGB, GscRGB, and BscRGB in the scRGBformat according to Equation 4.

5-2 Example of Monitor Display Processing Circuit 31 (Refer to FIG. 4)

The image data RscRGB, GscRGB, and BscRGB in the scRGB format before andafter the white balance control is converted into image data in the RGBformat in an scRGB/RGB conversion processor 311 in the monitor displayprocessing circuit 31, and the converted image data is supplied to theLCD 32 and is displayed as a color image. The conversion from the scRGBformat to the RGB format is described in detail next.

(1) The 16-bit linear image data RscRGB, GscRGB, and BscRGB in the scRGBformat is converted into the nonlinear image data R′scRGB, G′scRGB, andB′scRGB in the scRGB format according to Equations 5 and 6.(2) The image data converted in (1) is converted into the 8-bitnonlinear data R, G, and B in the RGB format according to Equation 18 inFIG. 25.(3) The 8-bit nonlinear image data R, G, and B converted in (2) issupplied to the LCD 32.

5-3 Example of GUI for Fine-Tuning AWB

An operation example of a GUI for fine-tuning the AWB is described withreference to FIG. 5. FIG. 5 includes diagrams showing the rear face ofthe digital still camera described above. On the rear face of thedigital still camera, the LCD 32, a menu button 41, an OK button 42, anda cursor button 43 for up-down and left-right movement are provided.

(1) An image for which the white-balance fine tuning is performed isselected and the selected image is displayed in the LCD 32.(2) The menu button 41, the cursor button 43, and the OK button 42 areoperated to select a “white-balance fine tuning mode”.

For example, as shown in FIG. 5A, letters indicating “Fine tuning”,“High”, “Standard”, and “Low” modes are superimposed and displayed onthe image displayed in (1). Among these modes, which can be selectedwith the cursor button 43 and the OK button 42, the “Fine tuning” isprovided for correction described below. When the “High” is selected,the white-point color temperature of the image is corrected to a valuehigher than the current value by 500 [K] to 1,000 [K]. When the “Low” isselected, the white-point color temperature of the image is corrected toa value lower than the current value by 500 [K] to 1,000 [K]. When the“Standard” is selected, the white-point color temperature of the imageis kept without correction.

(3) When the “Fine tuning” is selected, a slide bar 44 is furtherdisplayed in the LCD 32, as shown in FIG. 5B. When the cursor button 43is operated to horizontally move the slide bar 44, for example, thewhite-point color temperature of the image is corrected to a valuehigher or lower than the current value by around 100 [K] each time theslide bar 44 is moved by one scale division.

6. Example of Tone Correction Method in Tone Correction Circuit 35(Refer to FIG. 2)

The tone of 12-bit nonlinear image data YscYCC, CbscYCC, and CrscYCC inthe scYCC format, stored in the RAM 34, is corrected in the tonecorrection circuit 35. As described in detail below, the tone correctionis performed in accordance with GUI operations by the user.

6-1 Automatic Tone Correction

FIG. 6 includes diagrams illustrating examples of GUI operations in thetone correction.

(1) An image whose tone is to be corrected is selected and the selectedimage is displayed in the LCD 32.(2) The menu button 41, the cursor button 43, and the OK button 42 areoperated to select a “tone control mode”.

For example, as shown in FIG. 6A, letters indicating “Manual”,“Automatic”, “TV”, and “Picture” modes are superimposed and displayed onthe image displayed in (1). Among these modes, which can be selectedwith the cursor button 43 and the OK button 42, the “Manual”, isprovided for the user to manually perform the correction describedbelow. When the “TV” is selected, the image is corrected to achievetelevision-like image quality. When the “Picture” is selected, the imageis corrected to achieve film-camera-like image quality.

(3) When the “manual”, is selected, slide bars 45 and 46 are furtherdisplayed in the LCD 32, as shown in FIG. 6B. When the cursor button 43is operated to horizontally move the slide bar 45, the contrast of ahighlight of the image is corrected to a value higher or lower than thecurrent value by one step each time the slide bar 45 is moved by onescale division. When slide bar 46 is horizontally moved, the contrast ofa shadow of the image is corrected to a value higher or lower than thecurrent value by one step each time the slide bar 46 is moved by onescale division.(4) When the “Automatic” is selected in (2) described above, theautomatic tone correction is performed. The automatic correctionincludes, as shown in FIG. 6C, a “Standard” mode for general automaticcorrection, a “Nightscape” mode for the automatic correction for anightscape, and a “Snowscape” mode for the automatic correction for asnowscape. The user can select any of the “Standard”, “Nightscape”, andthe “Snowscape” modes.

6-2 Example of Tone Correction Circuit 35

FIG. 7 shows an example of the tone correction circuit 35. The tonecorrection circuit 35 is schematically described here, and thecomponents in the tone correction circuit 35 will be described in detailbelow. Referring to FIG. 7, among the image data YscYCC, CbscYCC, andCrscYCC stored in the RAM 34, the luminance data YscYCC is supplied to atone corrector 351. The tone of the luminance data YscYCC is correctedin the tone corrector 351 and the corrected data is output. The colordifference data CbscYCC and CrscYCC is supplied to a saturationcorrector 352. The saturation of the color difference data CbscYCC andCrscYCC is corrected in the saturation corrector 352 and the correcteddata is output. The output image data YscYCC, CbscYCC, and CrscYCC issupplied to LCD 32, as described above, and is displayed as a colorimage. The output image data YscYCC, CbscYCC, and CrscYCC is alsosupplied to the flash memory 20 and is stored in the flash memory 20.

The luminance data YscYCC is sequentially supplied to a luminance signalhistogram calculator 353, an image information extractor 354, and animage categorizer 355, and the images are categorized into, for example,ten categories. Based on the categorization result, the correctioncharacteristics of the tone in the tone corrector 351 are determined ina black-and-white-level correction curve generator 356, a tonecorrection curve generator 357, and a tone correction parameter selector367. In addition, based on the categorization result described above,the correction characteristics of the saturation in the saturationcorrector 352 are determined in a saturation correction curve generator358 and a saturation correction parameter selector 368. Variousparameters and thresholds are provided in a ROM 369. The followingprocessing is performed in components in the tone correction circuit 35.

6-3-1 Calculation for Making Cumulative Histogram of Luminance Signal Y

The luminance data YscYCC of the image for which the tone correction isto be performed is supplied from the RAM 34 to the luminance signalhistogram calculator 353. The luminance signal histogram calculator 353makes the cumulative histogram f(Y) of luminance signals Y from theluminance data YscYCC, as shown in FIG. 8.

6-3-2 Extraction of Image Information

The cumulative histogram f(Y) of the luminance signal Y, made in theluminance signal histogram calculator 353, is supplied to the imageinformation extractor 354. The image information extractor 354calculates values Y1, Y2, . . . Yn of the luminance signal Y when thedata of the cumulative histogram f(Y) corresponds to p1%, p2%, . . . pn% (for example, 5%, 10%, . . . 95%) of the entire data, as shown in FIG.8. The values Y1 to Yn represent the brightness of the image.

6-3-3 Categorization of Image

The image categorizer 355 uses the image information Y1 to Yn generatedin the image information extractor 354 to categorize the images into 12categories shown in FIG. 9. The image categorizer 355 performs thecategorization in accordance with, for example, an algorithm shown inFIG. 10.

(1) The luminance signal information value Y1 is compared with twothresholds AveLim1 and AveLim2, which is set in advance, to categorizethe brightness of the image into three categories; that is, “bright(Hi)”, “average (Ave)”, and “dark (Lo)”.(2) A value R (=Y3−Y2) is calculated from the luminance signalinformation values Y2 and Y3. The value R is compared with twothresholds RangeLim1 and RangeLim2, which is set in advance, tocategorize the range of the brightness of the image into threecategories; that is, “narrow (Narrow)”, “middle (Mid)”, and “wide(Wide)”. Hence, the image is categorized as any of a total of the ninecategories; that is, the three categories with respect to the brightnessof the image in (1)×the three categories with respect to the range ofthe brightness of the image.(3) An image having a “U-shaped” histogram, shown in the right side inFIG. 9, is extracted from the images in the three categories (bright,average, and dark) belonging to the “wide (Wide)” category with respectto the range of the brightness. This extraction is performed bycomparing the luminance signal values Y4 and Y5 of the image andgradient values S1 and S2 of the shadow and the highlight in thecumulative histogram with thresholds U-Lim1, U-Lim2, Slp1, and Slp2,which are set in advance, respectively.(4) The image to be corrected is categorized as any of the tencategories including the U-shaped histogram in the manner describedabove.(5) When the user selects a mode, such as the nightscape mode or thesnowscape mode, in the image capture, or when there are inputs with theGUI in the correction of the image (refer to FIG. 7), two categories of“Nightscape” and “Snowscape” are added, as shown in the right side inFIG. 9, based on the image capture information or based on the inputswith the GUI, respectively. In this case, the image is categorized asany of a total of 12 categories.

6-3-4 Black-and-White Level Correction

6-3-4-1 General Black-and-White Level Correction

The image data of the image categorized in the image categorizer 355 issupplied to the black-and-white-level correction curve generator 356, asshown in FIG. 7. The black-and-white-level correction curve generator356 primarily enhances an insufficient contrast caused by the exposurestate in the image capture. Accordingly, the black-and-white-levelcorrection curve generator 356 has an algorithm, for example, shown inFIG. 11 and has an S-shaped characteristic shown in FIG. 12.

The generation of this S-shaped characteristic uses functions inEquation 19 (refer to FIG. 25), in which an inflection point x0 and acurvature rr are used as parameters such that a luminance value Ymin ofthe black level of the image gets close to zero and a luminance valueYmax of the white level of the image gets close to 1.0. The inflectionpoint x0 and the curvature rr approximate a broken line given by drawinga straight line between the luminance value Ymin and the luminance valueYmax. The luminance values Ymin and Ymax ordinarily correspond to bothends of the histogram and are determined from the luminance signal valueY, which is a value appropriately extracted from the values in thecumulative histogram. However, a threshold YTH appropriate forpreventing overcorrection is set for the level Ymin at the black side.

6-3-4-2 Black-and-White Level Correction of “Nightscape” and “Snowscape”

As described above in 6-3-3, a correction effect different from that ofa category determined only from the cumulative histogram must beachieved in the “Nightscape” mode and the “Snowscape” mode. Hence,special processing is performed to correct the black-and-white level ina user scene selection corrector (refer to FIG. 11). The informationconcerning the “Nightscape” mode and the “Snowscape” mode is yieldedfrom the information input with the GUI by the user (refer to FIG. 7) orfrom the header information in the image file. The processing isdescribed next with reference to FIG. 13.

6-3-4-2-1 White Correction of Category “Nightscape”

The histogram of the nightscape is characterized by deviating to lowertones. It is often the case that the histogram of the nightscape has,for example, street light having a relatively small area and includesobjects having higher luminance values. In this case, as shown in thehistogram of the nightscape in FIG. 9, a certain number of pixels aredistributed near the maximum value of the tone. Accordingly, asufficient correction effect cannot be achieved at the white side byusing a method of determining the luminance value Ymax in normalblack-and-white correction.

In order to perform the correction effective for such a high-luminancearea, the white correction level for the nightscape is set to a valueslightly smaller than the luminance value Ymax set by a normal method.This setting shifts a light emitter, for example, a high-luminance areaincluding street light, toward the brighter area, thus effectivelyenhancing the luminance of the high-luminance area.

Without the setting described above, since the luminance value detectedfrom the cumulative histogram is decreased, the amount of correction tomake the luminance value close to the white value (1.0) becomes toolarge. In contrast, with the setting described above, since the value atthe midpoint between the detected luminance value and the white value(1.0) is used as the white correction level Ymax, it is possible toprevent the amount of correction from becoming too large. In addition,the darkness which the nightscape originally has is not derogated.

6-3-4-2-2 Black Correction of Category “Nightscape”

Since the tone of the nightscape is distributed in a lower range, it isdifficult to obtain the effect of the black correction itself by usingthe black level Ymin yielded by a normal method, and the effect of theblack correction level Ymin is not indispensable. However, it issupposed that the tone of the shadow can be slightly increased due tothe effect of the amount of correction at the white side. It is notpreferable to increase the tone of the shadow in a wider area, as in thenightscape, because the increased tone enhances noise in the imagecapture characteristics of current digital still cameras.

Hence, in order to surely suppress such enhancement of noise in thenightscape and aggressively enhance the darkness of the nightscape, theblack correction level Ymin is fixed to a value lower than the valueYmin yielded by a normal method to surely darken an area, having lowertones, in the shadow.

6-3-4-2-3 Black Correction of Category “Snowscape”

Since the pixels are deviated to higher tones in the snowscape, contraryto the nightscape, it is difficult to obtain the effect of the whitecorrection itself. However, it is supposed that the tone of thehighlight can be decreased due to the effect of the amount of correctionat the black side and the decreased tone of the highlight results in,for example, unnatural blackness such as a stain or a pseudo color inthe highlight.

Hence, in the snowscape, the black correction level Ymin is set to anappropriate value between the black level value Ymin yielded by a normalmethod and the value (0) of black to inhibit the effect of the blackcorrection. Even when the contrast is enhanced due to this blackcorrection, the whiteness characteristic of the snowscape is notderogated.

6-3-5 Tone Correction Parameter Selector 367 and Tone Correction

Objects of the tone correction here is to relatively enhance thecontrast of a range in which the tone is derogated mainly because ofinappropriate exposure and to return any excess effect of theblack-and-white level correction to a level appropriate for thecategory, as in the white level or black level correction describedabove. For example, an inverted S-shaped curve shown in FIG. 14 isapplied to such tone correction. Functions in Equation 20 (refer to FIG.25), having the inflection point x0 and the curvature rr as theparameters, are used to generate the inverted S-shaped curve.

Accordingly, the ROM 369 in FIG. 7 is provided with, for example, atable shown in FIG. 15 including the parameters of the inflection pointx0 and the curvature rr and saturation parameters kc described below forall of the twelve categories. The parameter x0 has values in a rangefrom 0.4 to 0.8. The parameter rr has values in a range from 1.0 to 10.

As shown in FIG. 16, the tone correction parameter selector 367 selectsthe corresponding parameter based on the category information on theimage output from the image categorizer 355 with reference to the tablein FIG. 15. The tone correction curve generator 357 generates theinverted S-shaped curve for the tone correction, as shown in FIG. 14, byusing the parameters selected in the tone correction parameter selector367.

Furthermore, the tone correction parameter selector 367 combines theinverted S-shaped curve for the tone correction with the S-shaped curve(FIG. 12) for black-and-white correction, generated in theblack-and-white-level correction curve generator 356, to generate acorrection translation table of the luminance signal values. The tonecorrector 351 uses the correction translation table generated in thetone correction parameter selector 367 to convert the image data on theluminance read out from the RAM 34 from a value Yin to a value Yout, asshown in FIG. 14, and to output the converted value Yout.

6-3-6 Saturation Correction

In a YCC color space, the saturation in a range from a middle-saturationarea to a high-saturation area can be derogated when the tone correctionis performed for the luminance signal channel as described above.Accordingly, correction for keeping the saturation is performed, alongwith the tone correction. This saturation correction is performed for achroma value C yielded from the color difference data Cb and Cr.Basically, a gain coefficient kc of the color difference data Cb and Cris controlled according to Equation 21 in FIG. 25 to enhance thesaturation.

Accordingly, the ROM 369 in FIG. 7 is provided with, for example, atable shown in FIG. 15 including the gain coefficients kc for all thetwelve categories. The parameter kc has values in a range from 1.0 to2.0.

As shown in FIG. 17, the saturation correction parameter selector 368selects the corresponding parameter kc based on the category informationon the image output from the image categorizer 355 with reference to thetable in FIG. 15. The saturation correction curve generator 358generates a correction curve, shown by a solid line in FIG. 18, based onthe straight line according to Equation 21 by using the parameterselected in the saturation correction parameter selector 368.

In this case, in order to avoid coloring a low-saturation area having analmost achromatic color, an appropriate threshold is set for thecorrection curve in FIG. 18, and the S-shaped functions according toEquation 19 are used to inhibit the saturation. An Hermite curve is usedin a high-saturation area in order not to clip values amplified in thesaturation enhancement. A correction translation table of the saturationdata Cb and Cr is generated based on this saturation correction curve.

The saturation corrector 352 uses the saturation correction tablegenerated in the saturation correction curve generator 358 to correctthe saturation data Cb and Cr, read out from the RAM 34, and to outputthe corrected data.

6-4 Tone Correction and Saturation Correction by User Selection

A case in which the user selects a “TV” correction mode or a “picture”correction mode by operating the GUI shown in FIG. 6 is described next.

As shown in FIGS. 19 and 20, the black-and-white level correction byusing the S-shaped curve, the tone correction by using the invertedS-shaped curve, and the saturation enhancement correction are performedin these modes, as in the automatic tone correction in 6-1. In thesecorrections, the amount of the black-and-white level correction by usingthe S-shaped curve and the amount of the saturation correction by usingthe gain coefficient are fine-tuned based on the amount of correction inthe normal automatic correction.

6-4-1 Image Quality Control in “TV” Mode

Images on TV screens (or images on CRT monitors) generally have higheraverage luminance, higher contrast (a sufficient black level and asufficient white level), and higher saturation. Accordingly, in order toobtain television-like images as a result of the image quality controlin the “TV” mode, the correction is performed in consideration of thesecharacteristics.

6-4-1-1 Black-and-White Level Correction

The black level correction is inhibited by comparing the black-and-whitelevel correction in the “Automatic” mode described in 6-3-4-1.Accordingly, a black level Ymin_TV in this mode is set according toEquation 22 in FIG. 25 using the black level Ymin determined in thenormal automatic correction. BKtv has a value in a range from 0.7 to1.0. A white level Ymax_TV in this mode is set according to Equation 23in FIG. 26 using the white level Ymax determined in the normal automaticcorrection. Wtv has a value in a range from 0.8 to 1.0.

After the processing described above is performed, as in the generationof the black-and-white level correction curve in the “Automatic”correction (FIG. 12), the inflection point x0 and the curvature rr ofthe S-shaped functions (Equation 19), which approximate a broken linegiven by drawing a straight line between the black level Ymin_TV and thewhite level Ymax_TV, are calculated in an S-shaped parameter calculator(refer to FIG. 19).

The image having a higher average luminance and a higher contrast iscaptured after the correction in the manner described above.

6-4-1-2 Tone Correction

As in the tone correction in the “Automatic” mode described in 6-3-5,the inverted S-shaped curve is generated based on the categorizedinformation on the image, the inverted S-shaped curve is combined withthe black-and-white level correction curve generated in 6-4-1-1, and theluminance data Y is corrected based on the correction curve resultingfrom the combination.

6-4-1-3 Saturation Correction

The saturation is further enhanced in the image quality control in the“TV” mode, compared with the correction based on the categorizedinformation on the image in the saturation correction in the “Automatic”mode described in 6-3-6. Accordingly, a gain coefficient kc_TV for thesaturation correction is calculated from the gain coefficient kc setbased on the categorized information on the image, according to Equation24 in FIG. 26. Gtv has a value in a range from 1.0 to 1.2.

The processing described above is performed in a user selectioncorrector in the saturation correction curve generator 358 in FIG. 20.Then, the correction curve is generated in the same manner as in thegeneration of the saturation correction curve in the “Automatic” mode.

6-4-2 Image Quality Control in “Picture” Mode

Picture images generally have higher contrast, but have lower averageluminance than that of the images on the TV screens. Accordingly, inorder to achieve film-camera-like image quality as a result of thecorrection in the “Picture” mode, the correction is performed inconsideration of these characteristics.

6-4-2-1 Black-and-White Level Correction

The black level is determined in the same manner as in theblack-and-white level correction in the “Automatic” mode described in6-3-4-1. A white level Ymax_Pic is calculated according to Equation 25in FIG. 26 using the white level Ymax determined in the “Automatic”mode. Wpic has a value in a range from 0.8 to 1.0.

After the processing described above is performed, as in the generationof the black-and-white level correction curve in the “Automatic”correction (FIG. 12), the inflection point x0 and the curvature rr ofthe S-shaped functions (Equation 19), which approximate a broken linegiven by drawing a straight line between the black level Ymin and thewhite level Ymax_Pic, are calculated in the S-shaped parametercalculator (refer to FIG. 19).

The image having a higher contrast and maintaining the halftones iscaptured after the correction in the manner described above.

6-4-2-2 Tone Correction

The tone correction is performed in the same manner as in the tonecorrection in the “TV” mode described in 6-4-1-2.

6-4-2-3 Saturation Correction

The saturation correction is basically performed in the same manner asin the saturation correction described in 6-4-1-3. The saturation isfurther enhanced in the image quality control in the “Picture” mode,compared with the correction based on the categorized information on theimage in the saturation correction in the “Automatic” mode described in6-3-6. Accordingly, a gain coefficient kc_pic for the saturationcorrection is calculated from the gain coefficient kc set based on thecategorized information on the image, according to Equation 26 in FIG.26. Gpic has a value in a range from 1.0 to 1.2.

The processing described above is performed in the user selectioncorrector in the saturation correction curve generator 358 in FIG. 20.Then, the correction curve is generated in the same manner as in thegeneration of the saturation correction curve in the “Automatic” mode.

6-4-3 Tone Control Mode by User

When the user selects the “Manual” in the GUI operation shown in FIG. 6,the user can control the contrast of the highlight in the image and thecontrast of the shadow in the image, as shown in FIG. 6B.

6-4-3-1 Black-and-White Level Correction

As shown in FIG. 6B, a black level Ymin_User of the image is correctedby using the slide bar 46 for controlling the contrast of the shadow. Inthis case, the black level Ymin_User is calculated according to Equation27 in FIG. 26 using the black level Ymin in the black-and-white levelcorrection in the “Automatic” mode described in 6-3-4-1. BKuser has avalue in range from 0.85 to 1.15. That is, the black level can becorrected, by using the slide bar 46, from 0.85 (the minimum contrast ofthe shadow) to 1.15 (the maximum contrast of the shadow) in incrementsof 0.05.

A white level value Ymax_User of the image is corrected by using theslide bar 45 for controlling the contrast of the highlight. The whitelevel Ymax_User is calculated according to Equation 28 in FIG. 26 usingthe white level Ymax in the black-and-white level correction in the“Automatic” mode described in 6-3-4-1. Wuser has a value in range from0.85 to 1.15 (however, when Ymax_User exceeds 1.0, Ymax_User=1.0). Thatis, the white level can be corrected, by using the slide bar 45, from0.85 (the maximum contrast of the highlight) to 1.15 (the minimumcontrast of the highlight) in increments of 0.05.

After the processing described above is performed, as in the generationof the black-and-white level correction curve in the “Automatic”correction (FIG. 12), the inflection point x0 and the curvature rr ofthe S-shaped functions (Equation 19), which approximate a broken linegiven by drawing a straight line between the black level Ymin_User andthe white level Ymax_User, are calculated in the S-shaped parametercalculator (refer to FIG. 19).

6-4-3-2 Tone Correction

The tone correction is performed in the same manner as in the tonecorrection described in 6-4-1-2.

6-4-3-3 Saturation Correction

The amount of the saturation correction is determined in accordance withthe amount of the black-and-white level correction calculated in6-4-3-1. Accordingly, a gain coefficient kc_User for the saturationcorrection by the user is calculated from the gain coefficient kc setbased on the categorized information on the image, according to Equation29 in FIG. 26. Guser has a value in a range from 0.85 to 1.15. The valueof Guser is determined according to Equation 30 in FIG. 26 using thevalues of BKuser and Wuwer, which vary with the control by the use ofthe slide bars 45 and 26 for the highlight and shadow in theblack-and-white level correction.

The processing described above is performed in the user selectioncorrector in the saturation correction curve generator 358 in FIG. 20.Then, the correction curve is generated in the same manner as in thegeneration of the saturation correction curve in the “Automatic” mode.

6-5 Monitoring of Image After Tone Correction (Refer to FIG. 4)

The images corrected in the manners described above are converted into8-bit nonlinear signals in the RGB format in a YcbCr/RGB conversionprocessor 312 in the monitor display processing circuit 31, and aresupplied to the LCD 32 and displayed as the images. The conversion inthe YcbCr/RGB conversion processor is performed according to a matrixoperation in Equation 31 in FIG. 26. M3-1 is the inverse matrix of thematrix used in Equation 7.

7. Features of Digital Still Camera Described Above

(1) Since the RAM 15, which stores the image data in the scene-referencecolor space format, that is, the 16-bit linear image data in the scRGBformat in the above description, is provided in the digital stillcamera, the user can control the white balance of the image only withthe digital still camera on site after the image capture without using apersonal computer or “image correction and processing software”.(2) Similarly, the user can correct the tone and saturation of the imageonly with the digital still camera on site after the image capture.(3) Even when the image is captured by another digital still camera,copying the image data in the flash memory 20 allows the white balanceof the image to be controlled owing to the provision of the RAM 15.(4) Similarly, even when the image is captured by another digital stillcamera, copying the image data in the flash memory 20 allows the toneand saturation of the image to be corrected.(5) Since the captured image is corrected based on the statisticalanalysis of the image when the tone and saturation of the image isautomatically corrected, the qualities of various captured images can beimproved with higher probability.(6) The correction curve generated by combining the S-shaped functionand the inverted S-shaped function is used in the tone correction, sothat the highlight of the image can be corrected independent of theshadow of the image to some extent.(7) Even a low-end user who does not have sufficient knowhow forcorrecting the tone or saturation of the captured image canautomatically correct a failed image with simple GUI operations.(8) High-end user who has a certain amount of knowhow for correcting thetone or saturation of the captured image can also simply correct theimage to his/her taste with GUI operations.(9) Television-like images or film-camera-like images according to thepreference of the user can be produced by the correction with simple GUIoperations.

8. Others

When the CCD image sensor 11 includes three CCD image sensorscorresponding to the three primary colors in the digital still cameradescribed above, the demosaic processing circuit 12 is not necessary.The flash memory 20 may be a removable memory card, such as a memorystick (registered trademark). Furthermore, the image data stored in theflash memory 20 may be output to an external device, such as a personalcomputer or a printer, through a USB or the like.

[List of Abbreviation Used in This Description]

AWB: Auto White Balance

CCD: Charge Coupled Device

CRT: Cathode Ray Tube

DCF: Design rule for Camera File Format

GUI: Graphical User Interface

JPEG: Joint Photographic Experts Group

LCD: Liquid Crystal Display

RAM: Random Access Memory

ROM: Read Only Memory

scRGB: relative SCene RGB color space

TV: TeleVision

USB: Universal Serial Bus

INDUSTRIAL APPLICABILITY

According to the present invention, since the memory, which stores theimage data in the scene-reference color space format, is provided in thedigital still camera, the user can control the white balance of theimage and/or can correct the tone and saturation of the image, only withthe digital still camera on site after the image capture. In addition,even when the image is captured by another digital still camera, thewhite balance, tone, and saturation of the image can be corrected.

Since the tone and saturation is automatically corrected based on thestatistical analysis of the image, the qualities of various capturedimages can be improved with higher probability. The correction curvegenerated by combining the S-shaped function and the inverted S-shapedfunction is used in the tone correction, so that the highlight of theimage can be corrected independent of the shadow of the image to someextent.

Even a low-end user who does not have sufficient knowhow for correctingthe tone or saturation can automatically correct the tone or saturationof a failed image with simple GUI operations. Furthermore, a high-enduser who has a certain amount of knowhow for correcting the tone orsaturation can also correct the image to his/her taste with GUIoperations. Television-like images or film-camera-like images accordingto the preference of the user can be produced by the correction withsimple GUI operations.

1.-19. (canceled)
 20. An imagining apparatus comprising: a firstconverting circuit that receives image data of an image subjected to anautomatic white balance correction and converts the image data accordingto one of a plurality of scene-reference color space formats; atemporary memory for storing the image data for which tone correction isto be performed in the one of the plurality of scene-reference colorspace formats; and a tone correction circuit, wherein the plurality ofscene-reference color space formats comprise a first scene-referencecolor space format and a second scene-reference color space formathaving linear image data and an extended color space of the firstscene-reference color space format, wherein the image data is read outfrom the temporary memory or a recording medium to perform the tonecorrection, and the image data resulting from the tone correction isrecorded in the recording medium.